Apache Pinot + Azure Event Hub Integration: Metadata Fetch Timeout
Hey team,
I’m integrating Apache Pinot (Dockerized) with Azure Event Hub (Kafka-compatible endpoint) for real-time ingestion. The connection works perfectly from a standalone Python Kafka client using confluent_kafka, but Pinot fails with:
org.apache.kafka.common.errors.TimeoutException: Timeout expired while fetching topic metadata
✅ Confirmed Working:
Pinot Server can curl and TLS handshake with icpeventhubns.servicebus.windows.net:9093
Python consumer successfully consumes data from ICPtrafficdataeventhub topic
Using correct sasl.jaas.config format with $ConnectionString and EntityPath
streamConfigs use stream.kafka.consumer.prop.* correctly
❌ Still Seeing:
Pinot throws a metadata fetch timeout when creating the real-time table, even though connectivity and credentials are confirmed.
Stream Config: For your reference
----------------------------------------
"streamConfigs": {
"streamType": "kafka",
"stream.kafka.topic.name": "icptrafficdataeventhub",
"stream.kafka.broker.list": "icpeventhubns.servicebus.windows.net:9093",
"stream.kafka.consumer.type": "simple",
"stream.kafka.consumer.factory.class.name": "org.apache.pinot.plugin.stream.kafka20.KafkaConsumerFactory",
"stream.kafka.decoder.class.name": "org.apache.pinot.plugin.stream.kafka.KafkaJSONMessageDecoder",
"stream.kafka.consumer.prop.bootstrap.servers": "XXXns.servicebus.windows.net:9093",
"stream.kafka.consumer.prop.security.protocol": "SASL_SSL",
"stream.kafka.consumer.prop.sasl.mechanism": "PLAIN",
"stream.kafka.consumer.prop.ssl.endpoint.identification.algorithm": "",
"stream.kafka.consumer.prop.sasl.jaas.config": "org.apache.kafka.common.security.plain.PlainLoginModule required username=\\\"$ConnectionString\\\" password=\\\"Endpoint=
sb://XXXns.servicebus.windows.net/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=My acess key=;EntityPath=icptrafficdataeventhub\\\";",
"realtime.segment.flush.threshold.rows": "50000",
"realtime.segment.flush.threshold.time": "24h"
}
#C013WKLT5T7 #C03N1JNHXLY